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ABSTRACT 


Although GSFC policy has required both a prototype (qualification 
model) and a flight (acceptance model) spacecraft, rising costs have 
caused a reexamination of this approach. In both the Interplanetary 
Monitoring Platform (IMP) and the Radio Astronomy Explorer (RAE) 
projects, a single spacecraft (termed proto-flight) was qualified and 
then successfully launched. The analysis discussed herein was con- 
ducted, using Monte Carlo techniques, to estimate whether this concept 
is truly cost-effective. The results of the study show a savings of 
approximately $1 million, using the proto-flight approach. It also shows 
that rigorous subsystem testing is effective in reducing costs if initial 
hardware quality is poor. The analysis reported herein assumes that 
each test approach treated produces equivalent spacecraft launch- 
readiness. Whether this assumption is true has not been determined; 
however, it could be expected to have some limitations. 
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TEST PLAN OPTIMIZATION FOR AN 
EXPLORER-SIZE SPACECRAFT 


by 

J. H. Boeckel and A. R. Timmins 

Goddard Space Flight Center 


INTRODUCTION 

The test philosophy at Goddard Space Flight Center has always emphasized complete 
spacecraft systems tests as a final check to demonstrate that the spacecraft is ready for launch. 
However, project managers have had considerable flexibility in conducting their programs. This 
has resulted in a minor amount of subsystem testing in some programs and substantial subsystem 
testing in others. In addition, the number of spacecraft in each program has varied considerably. 
Usually, a new program has included an engineering model, a prototype model, and two flight 
model spacecraft. In a subsequent launch in the same program, the number of spacecraft to be 
tested was reduced. As the funds available for a complete program declined, it was necessary to 
investigate ways to reduce costs. One approach was the use of a so-called ”proto-flight ff space- 
craft. This approach used a single spacecraft that was tested to prototype levels and subsequently 
launched. The Interplanetary Monitoring Platform (IMP) project personnel pioneered the use of a 
single spacecraft. However, questions on its performance relative to other test approaches could 
not be answered on the basis of only one trial. The present study provides information for judging 
comparative performance. 

The amount of hardware purchased is a basic determinant of program cost. Other factors 
such as amount and level of subsystem testing, combinations of subsystem and systems tests, de- 
lays, and levels of hardware quality also affect costs. Because of the effects of these factors, it 
became desirable to study different ways of conducting a test and evaluation program from the 
standpoints of cost and reliability, taking into account the expected variability in hardware quality. 
Because of the large number of interacting factors, such a study cannot be based upon a single 
program or a small number of spacecraft; experience with one or two specific projects does not 
provide an adequate guide concerning the costs involved. For adequate comparison, a large 
number of programs must be available for analysis. To provide such a basis, a mathematical 
cost model was developed in which the variables of cost, time, hardware quality, and test effective- 
ness could be studied, using large sample- sizes generated in a computer program, the General 
Purpose Systems Simulator (GPSS III). This program permitted the observation of total cost 
variation as caused by the random occurrence of failures and delays in a large number of "proj- 
ects” conducted under a given philosophy. 
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THE COST MODEL 


The GPSS in, as adapted for the present purpose, runs 2000 spacecraft programs using 
Monte Carlo techniques. Each step in the testing uses a probability distribution for selected 
failure rates. The output of the GPSS IE is a distribution of cost and time for each of the six 
phases of a project life cycle as follows: 

Phase I Development and Fabrication 

Phase II Subsystem Test 

Phase IE Integration 

Phase IV System Test 

Phase V Prelaunch 

Phase VI Space Flight 

The inputs to the cost model may be varied as desired. Details of the cost model are presented 
in Appendix A. 

This investigation concerns an Explorer-size spacecraft program, i.e., a spacecraft weighing 
up to approximately 500 pounds and launched by a Delta vehicle. Even with this limitation, one 
quickly recognizes that there is a considerable variation in total project costs within an Explorer 
program. This variation is accommodated by the cost model. 

Since historical cost data were not available in the form necessary for use in the model, the 
costs used are the best engineering judgments available with respect to each of the phases. 

Total program costs derived compare favorably with experience on typical explorer projects such 
as IMP F&G ($7.5 million each) and RAE-A ($8.1 million). 

SCOPE OF STUDY 

Table 1 lists failure data from the systems tests of six IMP spacecraft. It shows that the 
probability of failure of a particular subsystem ranged from 0.1 to 0.5. This result provides a 
basis for assuming the failure probability in systems tests in the model. 

Table 1 


Failure Data From Systems Tests of IMP Spacecraft. 



Number of failures 

Probability 

iivijr spacecraft 

Subsystems 

Experiments 

of failure P f 

s 

A - Prototype 

7 

4 

0.4 

B - Flight 

4 

1 

0.2* 

C - Flight 

1 

1 

0.1 

D - Flight 

8 

6 

0.5 

E - Flight 

5 

7 

0.4 

F - Proto-flight 

3 

3 

0.2 


*No. of subsystems 20 

No. of experiments 9 

Total units 29 


Example for P f - “ - 0.17 . 
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The variables studied included hardware quality, subsystem test effectiveness, subsystem 
and system test failure probability, and test plan. The range of each variable was as follows: 

1. Hardware quality: 0.25 to 0.95 failure probability 

2. Systems test failure probabilities: 0.07 to 0.90 

3. Subsystem test failure probabilities: 0.03 to 0.73 

4. Ratio of subsystem-to-system test effectiveness: 0.2 to 1.0 

5. Test plans: three 

Table 2 lists the matrix and specified values used for all the variables. One premise used in 
setting up Table 2 was that testing did not eliminate all failures in a spacecraft. A conservative 
space failure rate residual of 0.05 per subsystem was used. For instance, with a hardware failure 
probability of 0.95, the combined failure probability in subsystem and system testing was 0.90. 

Three cases were studied to examine the effect of the test plan on total project costs. Equal 
quantities of subsystem hardware were provided for each case. The three cases were as 
follows: 

1. Case I — Two complete sets of subsystems were tested as subsystems. One set was 
integrated into a flight spacecraft that was tested at prototype levels. No backup spacecraft was 
provided. 

2. Case H — No subsystem tests were performed. One prototype and one flight spacecraft 
were subjected to environmental tests. 

3. Case III — One set of subsystems was subjected to environmental tests and integrated into 
a prototype spacecraft. The flight spacecraft used subsystems that had not been subjected to 
environmental test. Both spacecraft systems were subjected to environmental tests. 

Subsystem test effectiveness was studied at three levels. 

1. Good — The subsystem test was made equal to the system test in effectiveness. 

2. Medium — The subsystem test was made one-half as effective as the systems test. 

3. Poor — The subsystem test was made one-fifth as effective as the systems test. 

As indicated in Table 2, this parameter was investigated at three levels under Case I only. 

The effect of systems test failure probability on the cost to launch was studied as a function 
of hardware quality and test plan. The matrix includes the effect of prior subsystem tests on the 
system test failure probability. 

Each computer run provides 25 tables of data. Each table gives the distribution of the results 
of 2000 spacecraft programs with respect to time, cost, or frequency of some part of the life 
cycle of the program. Approximately 20 individual cases were run for the purpose of this study. 

It is clear that many comparisons and analyses can be made from the available data. For 
example, test costs could be examined separately. 


3 



Table 2 


Summary of Variables Investigated. 


Hardware quality 
(P F = probability 
of failure) 


Test Plan 

Failure probability 
in test (P f ) 

ss = subsystem test; 
s = system test 

Case I 

Case II 

Case III 

Two sets of subsystems and 
one spacecraft (proto-flight) 

No. subsystem tests; 
one prototype and one 
flight spacecraft 

One set of subsystem tests; 
one prototype and one flight 
spacecraft 

Subsystem Test 
Effectiveness 

Prototype 

spacecraft 

Flight 

spacecraft 

Prototype 

spacecraft 

Flight 

spacecraft 

Good* 

Medium* 

Poor* 


p f 

0.73 

0.43 

0.18 


_ 

0.43 

- 

0.95 

s s 









Pf 

5 

0.17 

0.47 

0.72 

0.90 

0.30 

0.47 

0.30 


P f 

0.56 

0.33 

0.13 

- 

- 

0.33 

- 

0.75 

s s 









p f 

5 

0.14 

0.37 

0.57 

0.70 

0.23 

0.37 

0.23 


P f 

0.34 

0.21 

0.08 

- 

- 

0.21 

- 

0.50 

s s 









Pf 

5 

0.11 

0.24 

0.37 

0.45 

0.15 

0.24 

0.15 


P, 

0.13 

0.08 

0.03 

- 

- 

0.08 

- 

0.25 

s s 









p f 

s 

0.07 

0.12 

0.17 

0.20 

0.07 

0.12 

0.07 


*Good - Subsystem tests equal to systems tests in effectiveness 
Medium - Subsystem tests one-half as effective 
Poor - Subsystem tests one-fifth as effective. 



For the purpose of this study, the criterion used for comparison of the many variables is the 
cost to launch. This criterion is the most important one from the standpoint of the project 
manager since his interest is not served by minimizing costs of one phase of the program if it 
does not provide a savings for the overall program. 

RESULTS OF STUDY 

Subsystem Test Effectiveness 

Figure 1 presents data on the relationship of 
and the mean cost to launch. For this part of the 
were the basis for the cost data. With good 
hardware (failure probability, P f = 0.25), vari- 
ations in subsystem test effectiveness caused 
no significant difference in the mean cost to 
launch. With fair hardware (failure probability 
of 0.5), the difference between a good subsys- 
tem test and a poor subsystem test amounted 
to approximately $180,000 in the mean cost to 
launch. Inability to find faults at the subsys- 
tem level delayed their discovery until sys- 
tems tests were conducted. As the number of 
failures found in the more costly systems test 
increased, the mean cost to launch increased. 

With very poor hardware (failure probability 
of 0.95) the subsystem test effectiveness could 
make a difference of $600,000 in the mean cost 
to launch. 

Test Plan 

Subsystem test effectiveness compared to systems test effectiveness is influenced by the 
adequacy of system interface simulation as well as by the comparative duration of the two tests. 
The normal subsystem test time for a prototype subsystem test in an Explorer-type program is 
2 days for a thermal -vacuum test, compared to about 13 days (Reference 1) for a systems test. 
These considerations, in addition to a separate study of subsystem tests (Reference 2), resulted 
in using the "medium" subsystem test effectiveness in Figure 1 as being representative of the 
subsystem test effectiveness at Goddard Space Flight Center. 

Three test plans, identified as Case I, n, and III in Table 2, were investigated. Figure 2 shows 
the effect of hardware quality and test plan on the mean cost to launch. Case I is more economical 
under all circumstances investigated. With good hardware quality, Case I is more economical 


subsystem test effectiveness to hardware quality 
study, one set of subsystems and one spacecraft 


i.o, 
0-9 | 
0.8 | 
0.7 
0.6 
0.5 
0.4 
0.3 
0.2 
0.1 
0 



5.2 


4.0 4.2 4.4 4.6 4.8 5.0 

MEAN COST TO LAUNCH (millions of dollars) 

Note: GOOD - Subsystems tests equal to systems tests in effectiveness. 

MEDIUM - Subsystem tests half as effective. 

POOR - Subsystem tests one fifth as effective. 


Figure 1— Effect of hardware quality and subsystem test 
effectiveness on cost to launch, based on a test plan for 
one set of subsystems and one Explorer spacecraft. 


5 


than Case n or HI by $850,000, and the cost 
savings increase as the hardware quality de- 
creases. If hardware quality is very poor 
(failure probability of 0.95) the cost savings of 
Case I increases to $1.1 million compared to 
Case III and $1.6 million compared to Case n. 

Test Costs 


CASE I 

2 SETS OF SUBSYSTEMS 
1 FLIGHT SPACECRAFT 


S 0.8 

2 

o 

Of 

0.6 

LLJ 

OZ 

=> 

£. 0.4 


< 

£ 

o 

Q£. 

< 

X 


0.2h 



CASE III 
1 SET OF SUBSYSTEM TESTS 
1 PROTOTYPE AND 
1 FLIGHT SPACECRAFT 

I 

CASE II 

^ NO SUBSYSTEM TESTS 
1 PROTOTYPE AND 
1 FLIGHT SPACECRAFT 


6.0 


7.0 


8.0 


9.0 


10.0 


Two of the principal cost inputs for the 
three cases are the subsystem and system test 
costs. These costs were examined extensively 
as a function of test effectiveness and hard- 
ware quality, and the results were summarized 
in Figure 3. With very poor hardware, the 
need for a highly effective subsystem test is 
evident. In the most extreme case shown, the 

cost savings may amount to $1.75 million for an effectiveness ratio of 1.0 compared to 0.2. With 
good hardware, the cost savings of good subsystems testing becomes relatively small. 

Hardware Quality 


MEAN COST TO LAUNCH (millions of dollars) 

Figure 2— Effect of hardware quality and test plan on 
cost to launch. 


Figures 1 and 2 show decreased program costs for improved hardware quality. However, the 
model does not assign a cost penalty for improved quality. It is an independent variable and 
"comes for free TT . In reality, however, quality increases cost. There are two attacks on the 
quality problem: 

1. Standard quality assurance through workmanship, documentation, and design review. 

2. Pretesting under environmental stress before initiating the formal test program. 

Either of these approaches costs money. As indicated in Figure 2, the maximum benefit to 
be obtained through improved quality is approximately $1 million for this model program. There- 
fore, extra quality assurance and pretest are not cost-effective if they cost more than $1 million. 
However, one must recognize that some types of failures may escape detection by present test 
methods and can be eliminated only through quality assurance measures. 


Test Delays 

Another consideration in determining the best test plan is the time required to complete the 
program. Delays may be caused by late delivery of hardware, failures in subsystem and system 
testing, and unavailability of subsystems to complete integration or delivery to the launch site. 
These delays were combined for study as follows: 

1. All delays up to the start of systems tests 
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Figure 3-Effect of subsystem-to-system test effectiveness ratio on test costs for several levels of hardware 
quality, based on one set of subsystems and one spacecraft. 

2. Delays caused by system test failures 

3. Delays in shipping to the launch site because one or two subsystems had not arrived. 

The three types of delays are depicted in Figure 4 as a function of test effectiveness and hardware 
quality. Figure 4 shows: 

1. The delays preceding the start of system tests are not very sensitive to variation in hard- 
ware quality, and less sensitive to R (ratio of subsystem-to-system test effectiveness). 

2. The delay in system tests can vary significantly, depending principally on hardware 
quality. For a representative R value of 0.5, there could be 14 weeks of delay for very poor hard- 
ware, compared to 4 weeks for good hardware. 

3. The "good" subsystem test effectiveness ratio causes larger delays in shipping to the 
launch site. This results from the ground rule that system tests begin when 23 of 25 subsystems 
have been integrated. Thus, the system test may be completed before the last two subsystems 
have completed their subsystems tests. The chances of this happening increase with an increase 
in R. 
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Variation From Mean Costs 

The data presented thus far have been cited on the basis of the mean cost to launch. However, 
for a specific program, the cost to launch could vary from the mean, depending on chance. In 
order to make the best program decisions, a project manager needs to know how much chance 
alone might affect his program. The cost model provides such information by giving frequency/ 


cost distributions for each phase of the life cycle 



TEST EFFECTIVENESS: 

Good • Subsystem tesf equally effective as system tesK 

Medium * * ■ Subsystem test l/2 as effective as system test. 

Poor o o o Subsystem test l/5 as effective as system test. 

Figure 4— Summary of delays in a spacecraft program as a 
function of hardware quality and test effectiveness. 


Figure 5 is an example of such a distribution. 



COST TO LAUNCH (millions of dollars) 


Figure 5— Distribution of cost to launch for 2000 space- 
craft based on a test plan using one spacecraft and one 
set of subsystems (P f = 0.33, P f - 0.37). 


For instance, the cost to launch would not exceed $4.6 million (for the variables listed) for 50 per- 
cent of the time and would not exceed $5.20 million for 95 percent of the time. This kind of vari- 
ation exists even though the following variables were fixed: 

1. Test plan— one spacecraft and one set of subsystems 

2. System test failure probability— P f = 0.37 

s 

3. Subsystem test failure probability— P f = 0.33. 

ss 

The main point is that each program is unique, and we are trying to determine the probabilities 
for the cost to be within certain bounds. 
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Comparison of 50-Percent and 
95-Percent Values of Cost to Launch 

When mean values of cost to launch are used, the test plan identified as Case I is more eco- 
nomical in all cases. To gain further insight into the three test plans, the 50-percent values of 
the cost to launch were compared with the 95-percent values. Table 3 gives this comparison. It 
shows again that Case I is superior in all instances to Case II and Case m. In fact, the extreme 
values (95 percent values) for Case I are less than the mean values for Cases n and m. Further, 
Case I is affected less by hardware quality than Case II and Case in. Also, the cost to launch is 
* more sensitive to the test plan than it is to hardware quality. 

Table 3 

Cost To Launch In Thousands of Dollars. 




Mean values 

95% values 

Hardware 

p f 

Case no. 

Case no. 



V 

II 

III 

I * 

± B 

II 

III 

Quality 
Very poor 

0.95 

7726 

9324 

8819 

8291 

10470 

9891 

Poor 

0.75 

7525 

8839 

8539 

8090 

9978 

9502 

Fair 

0.50 

7310 

8342 

8240 

7892 

9439 

9295 

Good 

0.25 

7137 

8005 

7980 

7756 

9181 

9015 

Cost to 

0.95 

108 

131 

123 

116 

146 

138 

Launch 

0.75 

105 

124 

119 

112 ; 

139 ■ 

133 

as % of 

0.50 

103 

117 

115 

111 

132 

130 

$7137 

0.25 

100 

112 

112 

109 

128 

126 


*See Table 2 for identification of Cases Ig, II, and III. 


Relative Costs of Program Phases 

The relative costs (Table 4) are based on one spacecraft and one set of subsystems. The 
minimum and maximum values are taken from the spread of the mean values of the cases covered 
in this study. Using the foregoing representative figures, the phase I costs are shown to be greater 


Table 4 

Relative Costs of Program Phases. 


Phase 

Cost (Thousands of dollars) 

Minimum 

Maximum 

Representative 

I Research, development, and fabrication 

2500 

2700 

2600 

II Subsystem test ■ 

208 

427 

320 

III Integration 

380 

840 

610 

IV System test 

412 

1835 

1150 

V Prelaunch 

155 

225 

i 

190 
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than any other phase. These data are for one set of subsystems. All programs to date have used 
at least two sets of subsystems; thus a more representative value for phase I would be $5 million. 
The cost assumptions used in the model would have to be in error by orders of magnitude to 
change the conclusion that phase I offers the greatest potential for maximum cost savings. Another 
self-evident conclusion is that the cost of integrating a second spacecraft can be traded for testing 
two complete sets of subsystems. The subsystem and system test phases also offer cost savings 
possibilities, mainly through improvement of hardware quality. This analysis shows that the 
system test cost can be minimized by the use of a single spacecraft in conjunction with the com- 
plete testing of two sets of subsystems. 

CONCLUSIONS 

1. The cost model is useful in developing comparison data, using a large sample to investi- 
gate the effects of varying hardware quality, the test program plan, and test effectiveness. 

2. The cost model shows that the most economical way to conduct an evaluation program on 
an Explorer-type spacecraft is to use a test plan that requires one spacecraft with two sets of 
subsystems. 

3. The cost saving from an effective subsystem test program decreases as hardware quality 
improves. 

4. There is a clear limit to the cost savings that can be achieved by improving hardware 
quality prior to the spacecraft testing. 

FURTHER WORK 

As discussed earlier, it has been assumed throughout this study that all programs resulted in 
equivalent performance after launch. However, there are insufficient flight data to make a judg- 
ment. Therefore, it is desirable to investigate a range of possible situations. Such an investi- 
gation, however, would require an acceptable cost model for the impact of a failure in space to 
serve as a basis for comparison between programs. This phase of the analysis will require con- 
siderable development and input from scientists and managers. 

Goddard Space Flight Center 
National Aeronautics and Space Administration 
Greenbelt, Maryland, December 10, 1968 
124-12-03-01-51 
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Appendix A 


The Spacecraft Test Program Simulator 


Introduction 

Estimating the effect of various test philosophies on program cost requires a model of a 
spacecraft program. The following discussion describes the model chosen and the manipulations 
introduced. The mathematical modeling technique used is described in the General Purpose Sys- 
tems Simulator HI Users Manual (IBM Application Program H20-0163-1). With this tool, it was 
possible to simulate 2000 repetitions of each spacecraft program conducted with a given quality of 
hardware and under a given test philosophy but with variability in delay of hardware arrival, inte- 
gration, and test effectiveness. Thus variability in resultant cost could be observed for a given 
set of input data. 

The Spacecraft Program 

A spacecraft of the Explorer class was chosen for analysis. It consisted of approximately 25 
subsystems of which 11 were considered to be scientific experiments. Each subsystem was put 
through a simulated life-cycle of fabrication, integration, and testing by means of Monte Carlo 
techniques on a digital computer. (The Monte Carlo method consists of the use of random numbers 
to provide an artificial world of experience, based on assumed probability distributions for input 
variables, which is used for testing proposed operational schemes.) There are numerous possi- 
bilities at nearly every step in this life cycle. For example, a subsystem may fail in systems 
tests with a probability of 0.25. Therefore, an average of one-fourth of the subsystems in the 
spacecraft will fail. However, on a particular spacecraft, there is a finite (though small) proba- 
bility that all subsystems might fail or that none might fail, in accord with the laws of chance. 

Thus the results of a total program can vary widely, depending on chance although the input as- 
sumptions of arrival distributions and failure probabilities are held constant. To study this vari- 
ation, 2000 simulated programs were run for each set of input assumptions. The final results of 
these 2000 programs gave a distribution of possible costs related to the particular assumptions 
used. 

The model program emphasizes the test phase of the spacecraft program. Nominal durations 
for life cycle phases before the introduction of variability, failure, and delays are shown in 
Table Al. 
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A secondary output of the analysis, in ad- 
dition to program cost, is the delay in each of 
these phases as a function of the initial as- 
sumptions made on hardware quality, test ef- 
fectiveness, and test philosophy. 

Development/Fabrication 

The nominal time duration chosen for this 
phase was 40 weeks. However, experience 
shows that simultaneous, on-time arrival of 
all subsystems is very unlikely. The distri- 
bution chosen to represent the delay was log- 
normal, truncated at 50 weeks of delay. This means that no subsystem ever arrives ahead of 
schedule, the vast majority (85 percent) are no more than 20 weeks late, and none are later than 
50 weeks. It was assumed that disasters represented by schedule slippages of more than 1 year 
would be corrected by a dens ex mcichina that could not be interpreted mathematically. The 
actual distribution used had a mean delay of 12-1/2 weeks, and 99 percent of the items were in- 
cluded before the end of 50 weeks. 

The cost of each subsystem was computed at $2000 per week, uniformly for each subsystem. 
This cost would actually vary among subsystems during design and fabrication. However, it was 
felt that this variation would not have a significant effect on the interpretation of test philosophy 
changes. 


Table A1 

Project Life Cycle. 


Phase 

Nominal time (weeks) 

Development/fabrication 

40 

Subsystem test 

4 

Integration 

6 

System test 

8 

Prelaunch 

4 

Time to launch 

62 


Subsystem 

After a subsystem was delivered, it was subjected to subsystem testing, nominally a 4-week 
phase. These tests were not defined further; however, the failure probability (P f ) in these tests 

ss 

was a major input variable to the analysis. The P f is a function of two factors— the quality of 

s s 

the hardware as delivered and the effectiveness of the test. Hardware quality is defined as the 
probability (P f ) that the specific subsystem will fail catastrophically at some time before the 
spacecraft mission is complete. Test effectiveness is the ability of the test to find the inherent 
flaw, i.e., E s = P f /P f . When a subsystem failed, it was repaired (3 weeks) and retested (2 
weeks). The probability of failure in the second test varied also but was always set at one-third 
of the initial failure probability. When a second failure did occur, the subsystem was again re- 
paired (3 weeks) and retested (2 weeks). It was assumed that it always passed the third test. This 
truncation by guaranteed success does not conform to real life, but it was suitable for the purpose. 
Subsystems tests cost $2000 per week for each subsystem for both testing and repair time. The 
rate per week includes all project costs. 
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Integration 

I 

The nominal program calls for simultaneous delivery of all subsystems; however, as dis- 
cussed earlier, this does not happen. The model, therefore, begins integration when 13 subsystems 
are available and continues for 6 weeks or until 23 subsystems arrive. It is assumed that the 
23rd subsystem can be integrated in zero time. If the last two subsystems arrive during systems 
tests, no delay is charged; if they arrive after systems tests are complete, shipment to the launch 
site is delayed. This delay is charged at a rate of $40,000 per week, as is the integration itself. 
The rate per week, as in the case of the subsystem tests, includes all project costs. 

Systems Test 

Systems tests begin after integration is completed. The nominal time for these tests is 8 
weeks; however, their completion is usually delayed by failures. During systems tests, each sub- 
system is considered to have the same probability of failure P f . This value is variable in the 

s 

model. Its magnitude depends on initial hardware quality, the effectiveness of the prior subsystem 
tests in correcting faults, and the residual failure rate remaining in space. Thus system test 
effectiveness is given by 


p f 



In manipulating the model, the effectiveness of systems tests was always chosen as equal to, or 
greater than, subsystem test effectiveness. The ratio E ss /E s was varied from 1.0 to 0.2. The 
latter case represents rather cursory subsystem testing that would require the isolation of most 
faults at the systems test level. 

When failures did occur, they were categorized such that one-third required a retest of the 
system (Type A), and two-thirds were "quick-fix" without retest (Type B). Type A failures, which 
added 2 weeks each to the systems test time had a failure probability of P /2 in the retest for 

* s 

the particular subsystem in question. If another failure occurred, another 2-week delay was 
incurred, and the failure probability was P f /4 in the retest. If a failure occurred in this test, 
another 2-week delay was charged, but it was assumed that the fourth test was always successful. 
Type B failures (quick-fix) added 1 week each to the systems test time for repair but did not 
require retest. 

Systems testing was charged at a rate of $40,000 per week. As before, the rate per week 
includes all project costs. 


Shipment Delay 

As discussed earlier, the model permitted systems tests to be performed with two subsys- 
tems missing. However, shipping to the launch site was delayed until they arrived. This delay 
was charged at $40,000 per week. 
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Prelaunch 


The nominal time duration at the launch site was set at 4 weeks. Provision was also made 
for failures in this phase; they remained constant at 0.03. One week of delay was charged for 
each failure at a cost per week of $40,000. 


Space Flight 

The model provides for treating variable failure rates in space and a method of interpreting 
these failures in terms of cost. However, it is difficult to obtain a general agreement on this 
matter. In this analysis, it was assumed that every test philosophy analyzed produced equal re- 
sults in terms of finding inherent failures. Therefore, T, cost in space” was the same for each 
approach, and only "cost to launch" has been treated. Hopefully it will become possible to obtain 
agreement on the cost of failures in space and on the relative effectiveness of varying test 
philosophies so that the analysis can be extended to cover this aspect of total program cost. 
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